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[Abstract] 
[Object] 

To improve a compression ratio of an image by 
reducing the number of colors in a color-reduced image. 

15 [Solution] 

If the number of palettes of a character area has 
been 1 (S1901), a character area portion of a full-page 
binarized image is cut out to generate a one-portion 
binary image (SI 907) . It becomes a color-reduced image 

20 of the character area. On the other hand, if the 
number of the palettes is more than or equal to 2 
(S1901), a color reduction unit 1082 inputs an original 
image 101 and the full-page binarized image, sorts a 
value in the original image of a pixel which is black 

25 in the binary image within the character area, into a 
value of the palette, and generates the color-reduced 
image (S1902) . Then, a representative color is 
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extracted for each one character by using coordinate 
information on each one character according to the 
above described color-reduced image and character 
segmentation information (S1903), and character pixels 
5 in that one character area are redrawn with the 
representative color (S1904) . This process is repeated 
with respect to all character coordinates, and thereby 
a final color-reduced image is obtained. The above 
described process is performed with respect to all 
10 character areas (S1906) . 
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[What is Claimed IS: ] 
[Claim 1] 

An image processing apparatus characterized by 
comprising: 

5 character area detection means for, if a 

character area is included in an image, detecting said 
character area; 

ground image generation means for generating a 
ground image in which a character portion included in 
10 the . character area has been filled with a predetermined 
color; 

palette generation means for generating palettes 
of colors representing the character area; and 

color-reduced image generation means for, in a 

15 first color-reduced image of said character area, in 
which if the number of the palettes of the colors in 
the character area is one, an image in which said 
character area has been binarized has been set to the 
color-reduced image of said character area, and if the 

20 number of the palettes of the colors in the character 
area is multiple, the color-reduced image has been 
generated by using the palettes of the colors in said 
character area, generating a second color-reduced image 
of said character area by extracting a representative 

25 color in the character portion included in the 
character area of said first color-reduced image and 
drawing pixels constituting said character portion by 
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using said representative color. 
[Claim 2] 

The image processing apparatus according to claim 
1, characterized by further comprising binarization 
5 means for binarizing the image, 

wherein said character area detection means 
detects the character area included in the image by 
using a binarized image of said image generated by said 
binarization means. 
10 [Claim 3] 

The image processing apparatus according to claim 
1, characterized in that said character area detection 
means detects the character area included in said image 
from an image in which a differential process has been 
15 applied with respect to said image. 
[Claim 4] 

The image processing apparatus according to claim 
3, characterized in that said character area detection 
means detects the character area by applying a 
20 differential filter to said image to calculate an edge 
amount of all pixels with neighboring pixels and 
performing a contour tracing with respect to an image 
in which the edge amount has been binarized. 
[Claim 5] 

25 The image processing apparatus according to claim 

1, characterized by further comprising segmentation 
means for segmenting the image into areas of a 
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[Claim 6] 

The image processing apparatus according to claim 
1, characterized in that said ground image generation 
5 means further has average color calculation means for 
calculating an average color of colors in areas other 
than the character portion in the character area; and 

said character portion is filled by using the 
average color calculated by said average color 
10 calculation means. 
[Claim 7] 

The image processing apparatus according to claim 
1, characterized by further comprising JPEG compression 
means for JPEG-compressing the ground image generated 
15 by said ground image generation means. 
[Claim 8] 

The image processing apparatus according to claim 
7, characterized in that the ground image to be 
compressed by said JPEG compression means has been 
. 20 reduced. 

[Claim 9] 

The image processing apparatus according to claim 
l r characterized in that the ground image to be 
compressed by said JPEG compression means is reduced 
25 with a reduction ratio depending on areas having a 
large coefficient in a high frequency part when said 
ground image has been orthogonally transformed. 
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[Claim 10] 

The image processing apparatus according to claim 
1, characterized by further comprising color-reduced 
image compression means for compressing the color- 
5 reduced image of the character area generated by said 
color-reduced image generation means, with a 
compression method depending on the number of the 
palettes of the colors in said character area. 
[Claim 11] 

10 The image processing apparatus according to claim 

10, characterized in that if the number of the palettes 
of the colors in the character area is one, said color- 
reduced image generation means performs an MMR 
compression with respect to the color-reduced image of 

15 said character area, and if the number of the palettes 
of the colors in said character area is multiple, said 
color-reduced image generation means performs a ZIP 
compression with respect to the color-reduced image of 
said character area. 

20 [Claim 12] 

The image processing apparatus according to claim 
1, characterized in that, among the palettes generated 
by said palette generation means, a palette of a color 
which ' has not been used by said color-reduced image 

25 generation means is deleted. 
[Claim 13] 

An image processing method characterized by 
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comprising: 

a character area detection step for, if a 
character area is included in an image, detecting said 
character area; 
5 a ground image generation step for generating a 

ground image in which a character portion included in 
the character area has been filled with a predetermined 
color; 

a palette generation step for generating palettes 

10 of colors representing the character area; and 

a color-reduced image generation step for, in a 
first color-reduced image of said character area, in 
which if the number of the palettes of the colors in 
the character area is one, an image in which said 

15 character area has been binarized has been set to the 
color-reduced image of said character area, and if the 
number of the palettes of the colors in the character 
area is multiple, the color-reduced image has been 
generated by using the palettes of the colors in said 

20 character area, generating a second color-reduced image 
of said character area by extracting a representative 
color in the character , portion included in the 
character area of said first color-reduced image and 
drawing pixels constituting said character portion by 

25 using said representative color. 
[Claim 14] 

The image processing method according to claim 13, 
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characterized by further comprising a binarization step 
for binarizing the image, 

wherein said character area detection step 
detects the character area included in the image by 
using a binarized image of said image generated by said 
binarization step. 
[Claim 15] 

The image processing method according to claim 13, 
characterized in that said character area detection 
step detects the character area included in said image 
from an image in which a differential process has been 
applied with respect to said image. 
[Claim 16] 

The image processing method according to claim 13, 
characterized by further comprising a segmentation step 
for segmenting the image into areas of a predetermined 
size. 

[Claim 17] 

The image processing method according to claim 13, 
characterized in that said ground image generation step 
further has an average color calculation step for 
calculating an average color of colors in areas other 
than the character portion in the character area; and 

said character portion is filled by using the 
average color calculated by said average color 
calculation step. 
[Claim 18] 
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The image processing method according to claim 13, 
characterized by further comprising a JPEG compression 
step for JPEG-compressing the ground image generated by 
said ground image generation step. 
[Claim 19] 

The image processing method according to claim 13, 
characterized by further comprising a color-reduced 
image compression step for compressing the color- 
reduced image of the character area generated by said 
color-reduced image generation step, with a compression 
method depending on the number of the palettes of the 
colors in said character area. 
[Claim 20] 

The image processing method according to claim 19, 
characterized in that if the number of the palettes of 
the colors in the character area is one, said color- 
reduced image generation step performs an MMR 
compression with respect to the color-reduced image of 
said character area, and if the number of the palettes 
of the colors in said character area is multiple, said 
color-reduced image generation step performs a ZIP 
compression with respect to the color-reduced image of 
said character area. 
[Claim 21] 

The image processing method according to claim 13, 
characterized in that, among the palettes generated by 
said palette generation step, a palette of a color 
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which has not been used by said color-reduced image 
generation step is deleted. 
[Claim 22] 

A computer-readable storage medium storing 
5 program codes, characterized by comprising the program 
codes of: 

a character area detection step for, if a 
character area is included in an image, detecting said 
character area; 

10 a ground image generation step for generating a 

ground image in which a character portion included in 
the character area has been filled with a predetermined 
color; 

a palette generation step for generating palettes 
15 of colors representing the character area; and 

a color-reduced image generation step for, in a 
first color-reduced image of said character area, in 
which if the number of the palettes of the colors in 
the character area is one, an image in which said 
20 character area has been binarized has been set to the 
color-reduced image of said character area, and if the 
number of the palettes of the colors in the character 
area is multiple, the color-reduced image has been 
generated by using the palettes of the colors in said 
25 character area, generating a second color-reduced image 
of said character area by extracting a representative 
color in the character portion included in the 
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character area of said first color-reduced image and 
drawing pixels constituting said character portion by 
using said representative color. 
[Claim 23] 

5 The storage medium according to claim 22, 

characterized by further comprising the program code of 
a binarization step for binarizing the image, 

wherein said character area detection step 
detects the character area included in the image by 
10 using a binarized image of said image generated by said 
binarization step. 
[Claim 24] 

The storage medium according to claim 22, 
characterized in that said character area detection 
15 step detects the character area included in said image 
from an image in which a differential process has been 
applied with respect to said image. 
[Claim 25] 

The storage medium according to claim 22, 
20 characterized by further comprising the program code of 
a segmentation step for segmenting the image into areas 
of a predetermined size. 
[Claim 26] 

The storage medium according to claim 22, 
25 characterized in that the program code of said ground 
image generation step further has the program code of 
an average color calculation step for calculating an 



_ 11 ~ 



JPA2002-165105 



average color of colors in areas other than the 
character portion in the character area; and 

said character portion is filled by using the 
average color calculated by said average color 
5 calculation step. 
[Claim 27] 

The storage medium according to claim 22, 
characterized by further comprising the program code of 
a JPEG compression step for JPEG-compressing the ground 
10 image generated by said ground image generation step. 
[Claim 28] 

The storage medium according to claim 22, 
characterized by further comprising the program code of 
a color-reduced image compression step for compressing 
15 the color-reduced image of the character area generated 
by said color-reduced image generation step, with a 
compression method depending on the number of the 
palettes of the colors in said character area. 
[Claim 29] 

20 The storage medium according to claim 28, 

characterized in that if the number of the palettes of 
the colors in the character area is one, said color- 
reduced image generation step performs an MMR 
compression with respect to the color-reduced image of 

25 said character area, and if the number of the palettes 
of the colors in said character area is multiple, said 
color-reduced image generation step performs a ZIP 
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compression with respect to the color-reduced image of 
said character area. 
[Claim 30] 

The storage medium according to claim 22, 
characterized in that, among the palettes generated by 
said palette generation step, a palette of a color 
which has not been used by said color-reduced image 
generation step is deleted. 
[Detailed Description of the Invention] 
[0001] 

[Industrial Field Of Utilization] 

The present invention relates to an image 
processing apparatus which compresses an image, and a 
method thereof as well as a storage medium. 
[0002] 
[Prior Art] 

In recent years, scanners have become widely used 
and thereby documents have been progressively 
computerized. Retaining a computerized document in 
full color requires approximately 24 M bytes in the 
case of A4 size at 300 dpi, which causes tight memory 
conditions for retaining the document and is not a size 
capable of being sent to others as an attachment to a 
mail. 
[0003] 

On the other hand, JPEG is known for a full color 
image compression. Although the JPEG has a very high 
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effect and a good image quality in compression of 
natural images such as photos, image degradation called 
"mosquito noise" which JPEG-compresses a high frequency 
part such as a character portion is generated and a 
compression ratio is also poor. Consequently, first, 
an image is segmented into areas, and the JPEG 
compression is performed with respect to a ground 
portion from which a character area has been extracted. 
Moreover, an MMR compression or a ZIP compression is 
applied to a character area portion having color 
information. Thereby, images resulting from the 
respective compressions are obtained. 
[0004] 

On the other hand, there has been a 
representation method of transmitting the JPEG image at 
a white, portion and putting a representative character 
color on a black portion when these images are 
decompressed. 
[0005] 

[Problems That the Invention is to Solve] 

However,' in the conventional method, at an area 
determined as a character, a color reduction process 
has been performed with respect to the entire character 
area to generate a color-reduced image. Therefore, a 
color having a low proportion of being used in the 
character area is also extracted as the representative 
color, which has been a disadvantage of an increased 
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number of colors in the color-reduced image and a poor 

compression ratio. 

[0006] 

The present invention has been made in view of 
the above described problems, and it is an object of 
the present invention to improve the compression ratio 
of the image by reducing the number of the colors in 
the color-reduced image. 
[0007] 

[Means of Solving the Problems] 

In order to achieve the object of the present 
invention, for example, an image processing apparatus 
of the present invention is provided with a 
configuration as described below. In other words, the 
image processing apparatus of the present invention 
includes character area detection means for, if a 
character area is included in an image, detecting the 
above described character area, ground image generation 
means for generating a ground image in which a 
character portion included in the character area has 
been filled with a predetermined color, palette 
generation means for generating palettes of colors 
representing the character area, and color-reduced 
image generation means for, in a first color-reduced 
image of the above described character area, in which 
if the number of the palettes of the colors in the 
character area is one, an image in which the above 
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described character area has been binarized has been 
set to the color-reduced image of the above described 
character area, and if the number of the palettes of 
the colors in the character area is multiple, the 
color-reduced image has been generated by using the 
palettes of the colors in the above described character 
area, generating a second color-reduced image of the 
above described character area by extracting a 
representative color in the character portion included 
in the character area of the above described first 
color-reduced image and drawing pixels constituting the 
above described character portion by using the above 
described representative color. 
[0008] 

The image processing apparatus of the present 
invention further includes binarization means for 
binarizing the image, wherein the . above described 
character area detection means detects the character 
area included in the image by using a binarized image 
of the above described image generated by the above 
described binarization means. 
[0009] 

The image processing apparatus of the present 
invention further includes segmentation means for 
segmenting the image into areas of a predetermined size. 
[0010] 

Moreover,. the above described ground image 
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generation means further has average color calculation 
means for calculating an average color of colors in 
areas other than the character portion in the character 
area, and the above described character portion is 
5 filled by using the average color calculated by the 
above described average color calculation means. 
[0011] 

Moreover, the image processing apparatus of the 
present invention further includes JPEG compression 
10 means for JPEG-compressing the ground image generated 
by the above described ground image generation means. 
[0012] 

Moreover, the image processing apparatus of the 
present invention further includes color-reduced image 

15 compression means for compressing the color-reduced 
image of the character area generated by the above 
described color-reduced image generation means, with a 
compression method depending on the number of the 
palettes of the colors in the above described character 

20 area. 
[0013] 

[Embodiments of the Invention] 

Hereinafter, according to the accompanying 
drawings, the present invention will be described in 
25 detail according to preferred embodiments. 
[0014] 

[First Embodiment] 
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Figure 1 shows a configuration of an image 
processing apparatus of this embodiment for generating 
a compressed data 1A which will be described below, and 
processes (processing results) in its respective units. 
Reference numeral 101 denotes an original image. 
Reference numeral 102 denotes an image binarization 
unit which inputs the original image 101 and performs 
an optimal binarization of the image. Reference 
numeral 103 denotes a full-page binary image binarized 
by the image binarization unit 102. Reference numeral 
104 denotes a character area detection unit which 
inputs the full-page binary image 103, detects a 
character area in this image and generates character 
area coordinates 112. Reference numeral 105 denotes a 
character portion filling unit which extracts an area 
in the original image 101 positionally corresponding to 
a black area in the full-page binarized image 103, from 
the original image 101, fills the extracted area with a 
surrounding color and generates an image A. Reference 
numeral 106 denotes a reduction unit which inputs the 
image A and generates an image B by reducing the image 
A. Reference numeral 107 denotes a JPEG compression 
unit which inputs the image B and generates a 
compression code X 113 by JPEG-compressing the image B. 
Reference numeral 108 denotes a character color 
extraction unit which inputs the character area 
coordinates 112, calculates an original image color of 
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a black portion in the full-page binarized image 103 
while referring to the original image 101 and the full- 
page binarized image 103 within the coordinates, 
generates multiple palettes 114, and performs a color 
reduction process with respect to the original image 
101 according to the palettes 114. Reference numeral 
109 denotes color-reduced images of multiple character 
areas color-reduced by the character color extraction 
unit 108. Reference numeral 110 denotes an MMR 
compression unit which inputs the color-reduced images 
109 and generates multiple compression codes Y 115 by 
MMR-compressing the color-reduced images 109 when the 
color-reduced images 109 are 1 bit (m=l) . Reference 
numeral 111 denotes a ZIP compression unit which inputs 
the color-reduced images 109 and generates multiple 
compression codes Z 116 by ZIP-compressing the color- 
reduced images 109 when the color-reduced images 109 
are more than or equal to 2 bits (m>l) . Finally, data 
112 to 116 are integrated in 1A and this becomes the 
compressed data. 
[0015] 

Figure 3 shows a flowchart of a process until the 
character area is extracted from the original image 101. 
It should be noted that the original image 101 is 
assumed as a color image. 
[0016] 

At step S301, the original image 101 is inputted, 



- 19 - 



JPA2002-165105 



a luminance conversion is performed while reducing a 
resolution by thinning out the image, and a luminance 
image J is generated. For example, if the original 
image is 300-dpi, 24-bit RGB, an arithmetic operation 
5 of Y=0.299R+0.587G+0.114B is performed for every four 
pixels in both vertical and horizontal directions, and 
the image J having a luminance value Y is generated. 
, The image J becomes a 75-dpi, 8 -bit image. At step 
S302, a histogram of luminance data of the image J is 
10 formed and a binarization threshold T is calculated. 
[0017] 

At step S303, the luminance image J is binarized 
with the binarization threshold T, and the full-page 
binarized image 103 is generated. At step S304, a 

15 contour tracing is performed with respect to black 
pixels in the full-page binarized image 103, and all 
black areas are labeled. At step S305, character-like 
areas in the black areas are determined. In this 
determination, determination of the character-like 

20 areas (text blocks) is used according to a block 
discrimination method described in Japanese Patent 
Laid-Open No. 06-068301. 
[0018] 

Next, at step S306, the areas which should be 
25 integrated are integrated to each other in accordance 
with shapes or positions of the black areas. Moreover, 
at this step, the coordinates of the respective 
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character areas and the coordinates of the area for 

each one character have also been detected. 

[0019] 

An example of the above described process will be 
shown. For example, if a color manuscript shown in 
Figure 4 is inputted, the luminance conversion is 
performed while thinning out the color manuscript, and 
the histogram thereof is formed, the histogram becomes 
as shown in Figure 5. If the threshold T=150 is 
calculated by using data such as an average and a 
variance from this histogram, and the image shown in 
Figure 4 is binarized by using this threshold T, the 
binarized image (the full-page binarized image 103) 
becomes as shown in Figure 6. If the contour tracing 
is performed with respect to the black pixels in Figure 
6 to label all, and for example, only collections of 
the black pixels having widths less than or equal to a 
predetermined threshold or heights less than or equal 
to a predetermined threshold are permitted as the 
characters, the collections of the black pixels shown 
in Figure 7 become the character areas (it is just 
imagery and such an image is not actually generated) . 
[0020] 

If necessary, when these collections of the black 
pixels are grouped in accordance with positional 
proximity or matching widths or heights, 17 character 
areas can be detected as shown in Figure 8. Coordinate 
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data on these character areas is the character area 

coordinates 112 in Figure 1. 

[0021] 

Next, an example of a process in the character 
5 portion filling unit 105 by using the original image 
and the full-page binarized image (binary image) will 
be described by using respective images shown in Figure 
10 and a flowchart of the same process shown in Figure 
11. Figure 10{a) is the original image which will be 
10 used below. It is assumed that the binary image of one 
character area shown in the same Figure (b) has been 
obtained from this original image. 
[0022] 

At step S1101, the original image is segmented 
15 into 32x32 areas (hereinafter referred to as parts) , 
and the process is performed for each part. Figure 
10(c) shows how the original image has been segmented 
into parts. With respect to 6 parts 00 to 10, since 
there is no character area at a branch of step S1103, 
20 no process is performed. With respect to a part 11, 
since the character area is included, the process 
proceeds to step S1104. At step S1104, with reference 
to an area of the binary image corresponding to the 
part 11, an average value (average color) ave_color of 
25 RGB values (or may be any values such as YUV) of the 
color image corresponding to a white portion of this 
area is calculated. At step S1105, at this time, with 
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reference to the corresponding binary image, the above 
described ave_color is assigned to the color image 
corresponding to the black pixels. The above process 
is repeated with respect to parts 12, 13, 21, 22 and 23 
in which the character areas exist. With respect to 
other parts, since there is no character area, no 
process is performed. In this way, the portions where 
the characters have existed have been able to be filled 
with the average value of neighboring pixels. 
[0023] 

This image (image A) is reduced by the reduction 
unit 106. In this embodiment, the reduction is 
performed by a simple thinning. Incidentally, an order 
of this reduction process and the character portion 
filling process may be reversed. .In that case, 
consideration must be given to any positional shift 
between the binary image and the color image. 
[0024] 

Next, a flowchart of a process in the character 
color extraction unit 108 is shown in Figure 12. 
Although the full-page binary image 103 has been used 
as the binary image, the binary image is not limited 
thereto, and for example, only the coordinates of the 
character areas and the color image are inputted, the 
color image is binarized again, and its result may be 
used to perform a process of calculating a 
representative color as will be described below. 
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[0025] 

At step S1201, a counter num indicating the 
number of extracted colors is reset to 0. Since the 
process is performed with respect to each of all 
5 extracted character areas, it is checked whether or not 
there are any unprocessed character area coordinates at 
step S1202, and if there are the unprocessed character 
area coordinates, the process proceeds to step S1203, 
and if there are no unprocessed character area 
10 coordinates, the process is terminated. 
[0026] 

At step S1203, a thinning process is performed 
with respect to the binary image corresponding to the 
character area coordinates so as to reduce black 

15 corresponding to a transitional portion from a ground 
to a character portion, and a new binary image newbi is 
generated. Next, at step S1204, an RGB 3D histogram of 
the original image corresponding to the black pixels of 
the new binary image newbi is formed. On this occasion, 

20 if the histogram is normally formed, for example, when 
the inputted image is assumed as 8-bit RGB, a histogram 
of 256x256x256 is required. Considering that the 
resolution is required for the Character portion but 
tone is not required, and also that it is better to 

25 ignore some difference in pixel values in the 
calculation of the representative color while reducing 
variation in reading by a scanner and the like, such a 
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detailed histogram is not required. Therefore, in this 
example, the RGB 3D histogram of high 5 bits is formed. 
When this histogram is formed, a total number of the 
black pixels existing in the character area, pixelnum 
5 is also calculated. It should be noted that, although 
an RGB space has been used in this embodiment, another 
color space such as YUV may be used. Moreover, 
although the 3D histogram has been formed, three ID 
histograms for the respective colors may be formed. 
10. [0027] 

At step S1205, a maximum value is calculated from 
the RGB 3D histogram. Although the histogram of only 
high 5 bits has been already formed and a noise due to 
the variation in the scanner, has been reduced in this 

15 example, it is possible to detect an original maximum 
value across two histograms as shown in Figure 13 by 
calculating a maximum value of a sum of adjacent values 
in the histograms. Specifically, since it is the 3D 
histogram, it is conceivable that the maximum value of 

20 a total value of a total of seven histogram values, 
that is, a histogram value at a noted point, two 
histogram values adjacent in an R axis direction, two 
histogram values adjacent in a G axis direction, and 
two histogram values adjacent in a B axis direction is 

25 detected and the like. The maximum value detected in 
this way is assigned to colR[num], colG[num], colB[num]. 
[0028] 
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At step S1206, the histogram values positioned 
within a square which has been enlarged, for example, 
by 3 steps (=haba) respectively centering around the 
maximum value detected at step S1205, are set to 0. A 
description of the square is shown in Figure 15. 
Figure 15 shows the 3D histogram, and it is assumed 
that a portion denoted by a black dot is colR[num], 
colG[num], colB[num]. A total of 7x7x7 which has been 
enlarged by 3 steps respectively centering around the 
point is the above described square. Here, 3 steps 
correspond to 24 levels in 256 tones since it is the 
histogram of high 5 bits. This is only an example and 
the steps are not limited thereto. After the histogram 
values within this square are subtracted from pixelnum, 
0 is assigned. At step S1207, the above described 
variable num is incremented by one. 
[0029] 

At step S1208, it is checked whether or not 
pixelnum is more than or equal to a previously decided 
threshold threl, and if pixelnum is more than or equal 
to threl, the process proceeds to step S1205, and if 
pixelnum is less than threl, the process proceeds to 
step S1202. 
[0030] 

The above process is repeated with respect to all 
character areas, and thereby the palettes of all 
character areas are generated. Next, a flowchart of a 
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process in a color reduction unit 1082 within the 
character color extraction unit 108 is shown in Figure 
19 and will be described below. 
[0031] 

5 If the number of the palettes of the character 

area has been 1 (step S1901), a character area portion 
of the inputted full-page binarized image 103 is cut 
out to generate a one-portion binary image (step S1907) 
It becomes the color-reduced image of the character 
10 area. 
[0032] 

On the other hand, if the number of the palettes 
of this area is more than or equal to 2 (step S1901) , 
the color reduction unit 1082 inputs the original image 

15 101 and the full-page binarized image 103, and first, 
without using character segmentation information, sorts 
the value in the original image of the pixel which is 
black in the binary image within the character area, 
into the value of the palette, and generates the color- 

20 reduced image (step S1902) . Then, the representative 
color is extracted for each one character by using 
coordinate information on each one character according 
to the above described color-reduced image and the 
character segmentation information (step S1903), and 

25 character pixels in that one character area are redrawn 
with the representative color (step S1904) . 
[0033] 
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It should be noted that here the character 
segmentation information is obtained by a method 
disclosed in Japanese Patent Laid-Open No. 06-068301, 
and the method is a method of obtaining the coordinate 
information on each one character by a character cutout 
process. Specifically, a line segmentation process and 
the character cutout process are performed with respect 
to a block determined as a text area, and the 
coordinate information (character segmentation 
information) on each one character is obtained. 
[0034] 

This process is repeated with respect to all 
character coordinates, and thereby the color-reduced 
image 109 is obtained. Among the generated palettes, a 
palette color which has not been used in the color- 
reduced image 109 is deleted from the palettes (step 
S1905) . With respect to an allocated number of bits, 
since 1 data is required for indicating transparence as 
a white portion in the binary image, when the number of 
the palettes is 3, it becomes 4 by adding 1 
transparence data and the allocated number of bits 
becomes 2 bits, and when the number of the palettes is 
4, it becomes 5 by adding 1 transparence data and the 
allocated number of bits becomes 3 bits. This number 
of bits is in accordance with the number of bits 
capable of being represented in an image format which 
is finally adapted. On this occasion, for example, if 
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the number of the palettes is significantly large and 
it can be determined that effects of improving an image 
quality and improving a compression ratio due to 
multicoloring cannot be seen, it is also conceivable 
that the multicoloring is not performed and the image 
is saved as a ground image. In that case, the 
character area has to be deleted from the character 
area coordinate information 112. Moreover, it has to 
be deleted from the character area coordinate 
information before the process in the character portion 
filling unit 105 is performed. The process according 
to the flowchart shown in Figure 19 is performed with 
respect to all character areas (step S1906) . 
[0035]. 

If the color-reduced image 10 9 generated in this 
way is 1 bit, the color-reduced image 109 is MMR- 
compressed in the MMR compression unit 110 and the 
compression code Y is generated. Moreover, if the 
color-reduced image 109 is more than or equal to 2 bits, 
the color-reduced image 109 is ZIP-compressed in the 
ZIP compression unit 111 and the compression code Z is 
generated. On the other hand, the JPEG compression is 
performed with respect to the reduced image B, and the 
compression code X is generated. A format (compressed 
data 1A) which have integrated five elements, that is, 
the character area coordinates 112, the palettes 114, 
the compression code X 113, the compression codes Y 115 
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and the compression codes Z 116 is generated if 

necessary. 

[0036] 

It should be noted that, for example, if all 
5 character areas are the binary images, since the color- 
reduced images generated in the character color 
extraction unit 108 are 1 bit, the compression codes Z 
116 are not generated in this case. In other words, 
the data included in the compressed data 1A is not 

10 always limited to the above five elements, and the data 
may be four elements as described above, depending on 
the character areas. The same also applies to, for 
example, the case where all character areas are the 
color images. As an example of the format integrating 

15 the five elements, PDF of Adobe or the like is 
conceivable. The PDF of Adobe is a format that can be 
displayed by a free application called Acrobat Reader 
distributed by Adobe. This can prevent a trouble such 
as inability of a reception side to open a file because 

20 of absence of an application that was used to generate 
a document.. As other formats, XML and the like are 
available. XML is a description language for 
exchanging or distributing documents or data through a 
network. 

25 [0037] 

Figure 2 shows a configuration of the image 
processing apparatus of this . embodiment for 
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decompressing the compressed data 1A and also processes 

in its respective units. 

[0038] 

Reference numeral 201 denotes JPEG decompression 
means which inputs the compression code X 113 and 
performs a JPEG decompression process to generate a 
multivalued image E. Reference numeral 202 denotes an 
. enlargement unit which inputs the multivalued image E 
and performs an enlargement process. Reference numeral 
203 denotes a multivalued image F enlarged by the 
enlargement unit 202. 
[0039] 

Reference numeral 204 denotes an MMR 
decompression unit which inputs the compression codes Y 
115 and generates binary images G 205. Reference 
numeral 206 denotes a ZIP decompression unit which 
inputs the compression codes Z 116 and generates 
multicolor images H 207. Reference numeral 208 denotes 
an image integration unit which inputs the character 
area coordinates 112 and the corresponding palettes 114, 
as well as the binary images G 205 and/or the 
multicolor images H 207, and if pixel data of the 
binary images or the multivalued images represents the 
transparence, selects colors of the pixels of the 
multivalued images F 203, or otherwise selects the 
corresponding palette colors, and generates an image I 
(decompressed image) 209 which is a final image. 
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[0040] 

Figure 14 shows a process of the integration 
processing unit 208. First, Figure 14(a) shows a 
result of JPEG-decompressing the compression code X. 
Although the image of Figure 10 has been used for this, 
the data has the pixel values slightly different from 
Figure 10 (c) by using an irreversible quantization 
method of the JPEG compression. However, variation in 
the pixel values is smaller in the case of using the 
same quantization table, in comparison with the case of 
compressing the original image before its character 
portions are extracted, by a JPEG irreversible 
compression method. In this example, the image 
included in the character area (a result of 
decompressing the compression codes Y 115) is 
represented by 1 bit of the binary image as shown in 
Figure 14(b), and its palette is assumed as R=20, G=30, 
B=225. With reference to the binary image shown in 
Figure 14(b), data on the palette color (20, 30, 255) 
is put on portions corresponding to the black pixels on 
the image shown in Figure 14(a), and finally an image 
as shown in Figure 14(c) is generated. This becomes 
the decompressed image 209. If the image included in 
the character area is the multicolor image, the number 
of the palettes is changed. For example, in the case 
of 2 bits, the palettes allocated to four pixel values 
of 00, 01, 10 and 11 are applied. Among them, one 
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pixel value indicates the transparence, and for example, 
if it is 00, the pixel having the value of 00 selects 
the pixel of the image shown in Figure 14 (a) . 
[0041] 

5 As described above, according to the image 

processing apparatus and the method thereof in this 
embodiment, if the number of the palettes within the 
character area is 1, this character area is represented 
by using the binary image. On the other hand, if the 

10 number of the palettes is more than or equal to 2, the 
respective characters are represented with the 
respective representative colors. As a result, colors 
having a low proportion of being used in the character 
area are not included in the color-reduced image, and 

15 therefore an image having a good compression efficiency 
with less number of colors can be generated. 
[0042] 

[Second Embodiment] 

Although the original image (color image) has 

20 been binarized as the character area extraction process 
in the first embodiment, the process is not limited 
thereto and there is a method of applying a 
differential filter to the original image, calculating 
an edge amount of all pixels with neighboring pixels, 

25 binarizing the edge amount to obtain the binary image, 
and similarly performing the contour tracing with 
respect to the binary image to detect the character 
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areas. A configuration of the image processing 
apparatus which executes this method will be as shown 
in Figure 16. 
[0043] 

5 In the configuration example of Figure 16, since 

a differentially binarized image 1703 cannot be used 
for the character portion filling and the character 
color extraction, a binary image (partial binary image 
1706) is generated for each character area. This 

10 binary image may be binarized, for example, with the 
threshold T calculated by a character area detection 
unit 1704, or a histogram is formed for each area and 
an optimal binarization threshold may be calculated 
depending on the character area. In comparison with 

15 the full-page histogram shown in Figure 5, a luminance 
histogram of one portion of the character area can be 
expected to be a simple form as shown in Figure 9, and 
thereby the threshold is easily determined. Reference 
numeral 901 denotes a collection of ground colors, and 

20 reference numeral 902 denotes a collection of character 
colors . 
[0044] 

In the first embodiment, a character having a 
high luminance on a ground having a low luminance 
25 (reversed character) cannot be processed. However, if 
the character area is detected by this differential 
process, this reversed character area can also be 
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detected. Reference numeral 1702 denotes a 

differential processing unit, which applies a 
differential filter as shown in Figure 17 centrally 
around a noted pixel and performs the binarization so 
that the pixel is black if its absolute value exceeds 
the threshold or the pixel is white if its absolute 
value does not exceed the threshold. Figure 17(a) 
shows a linear differential filter, in which an upper 
filter can detect horizontal lines and a lower filter 
can detect vertical lines. Hatched lines can be 
detected by using a sum of absolute values of two 
filters. Moreover, a filter for the hatched lines may 
be used. Figure 17(b) shows a quadratic differential 
filter which accommodates all directions. Also in the 
quadratic differential filter, a horizontal direction 
filter and a vertical direction filter can be generated. 
Such a filter is applied to all pixels to generate the 
differentially binarized image 1703. At this time, the 
resolution can also be simultaneously reduced by 
applying the filter while thinning out the pixels. If 
the process from step S304 of Figure 3 is performed 
with respect to the binary image generated as described 
above, the character area coordinates including the 
reversed character can be detected. 
[0045] 

In a configuration directed also to this reduced 
character, a binarization unit 1705 also has to 
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accommodate the reversed character. In the first 
embodiment, it has been assumed that only the pattern 
shown in Figure 9 is entered. However, in the case of 
this configuration in which the reversed character area 
can also be extracted as the character area, three 
patterns shown in Figure 18 will be mainly entered. 
Figure 18(b) is a diagram showing a luminance 
distribution of the character and the ground in the 
case of the reversed character. Moreover, Figure 18(c) 
is a diagram showing the respective luminance 
distributions in the case where there are two colors of 
a black character and a white character on the same 
gray ground. In consideration of these three patterns, 
the binarization unit 1705 may detect a point A and a 
point B, and perform the binarization process so that 
an area between A and B is white and other areas are 
black. Alternatively, without considering the pattern 
shown in Figure 18 (c) , one threshold for dividing the 
ground and the character portion may be detected, and a 
process of performing a reversal in the case of a 
reversed pattern may be performed. 
[0046] 

If the reversed character area is also 
accommodated in this way, the reversed character area 
which has remained on the JPEG-compressed image in the 
first embodiment is also smoothed by the character 
portion filling, which also provides the good 
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compression efficiency and also enables its reversed 
character portion to be compressed without degradation 
in the resolution or degradation due to a mosquito 
noise. 
5 [0047] 

Moreover, although the character area extraction 
process has been performed by using the binary image in 
the first embodiment and this embodiment which uses the 
above described differentiation, the process is not 
10 limited thereto, and the character area may be inferred 
with reference to the multivalued image's own pixel 
values . 
[0048] 

In the first embodiment, a degree of the 
15 reduction in the reduction unit 106 has been constant 
in any image. However, it is not limited thereto, and 
for example, a reduction parameter determination unit 
which determines a reduction parameter for a resolution 
conversion (for example, one-half, one-quarter and the 
20 like) may be provided. As a method of realizing this, 
for example, an orthogonal transformation is performed 
for each 8x8 on a full page of the image A, and the 
reduction can be adjusted so that the reduction is by 
one-half if the number of existing areas having a large 
25 coefficient in a high frequency part of a result of the 
orthogonal transformation is more than or equal to a 
threshold, the reduction is by one-quarter if it is 
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less than or equal to the threshold, or the like. This 
parameter is not limited to two stages, and for example, 
the parameter can also be three stages (no reduction, 
one-half and one-quarter) . This avoids an extreme 
5 reduction in the high frequency part and provides an 
effect of avoiding degradation in the image quality. 
As this reduction parameter determination, a method of 
applying the differential filter to the image and 
switching according to a summation of its absolute 

10 values is also conceivable. For example, it is 
conceivable that the reduction is not performed if a 
summation of differences between adjacent pixel values 
is more than or equal to a threshold m, the reduction 
is by one-half if it is more than or equal to n, the 

15 reduction is by one-quarter if it is less than n, or 
the like. 
[0049] 

It should be noted that a configuration of an 
apparatus for decoding the compressed data 1A generated 
20 by the image processing apparatus in this embodiment 
may be the apparatus having the configuration shown in 
Figure 2 . 
[0050] 

[Other Embodiments] 
25 In the above described embodiments, the number of 

tiles is not required to be multiple as shown in Figure 
2 or Figure 5, and the entire image may be processed as 
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one tile. In this case, it is not necessary to extract 

a characteristic amount for each tile. 

[0051] 

Furthermore, the present invention is not limited 
5 only to the apparatus and the method for realizing the 
above described embodiments, and the case where 
software program codes for realizing the above 
described embodiments are supplied to a computer (a CPU 
or an MPU) in the above described system or apparatus 

10 and the computer in the above described system or 
apparatus causes the above described various devices to 
operate according to these program codes to realize the 
above described embodiments is also included in the 
category of the present invention. 

15 [0052] 

Moreover, in this case, the above described 
software program codes themselves realize functions of 
the above described embodiments, and the program codes 
themselves and means for supplying the program codes to 
20 the computer, that is, specifically, a storage medium 
storing the above described program codes are included 
in the category of the present invention. 
[0053] 

As the storage medium storing the program codes 
25 as described above, for example, a floppy (registered 
trademark) disk, a hard disk, an optical disk, a 
magneto optical disk, a CD-ROM, a magnetic tape, a 
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nonvolatile memory card, a ROM and the like can be used. 
[0054] 

Moreover, not only in the case where the above 
described computer controls the various devices only 
5 according to the supplied program codes and thereby the 
functions of the above described embodiments are 
realized, but also in the case where the above 
described program codes collaborate with an OS 
(Operating System) running on the computer or other 
10 application software and the like and thereby the above 
described embodiments are realized, such program codes 
are included in the category of the present invention. 
[0055] 

Furthermore, the case where these supplied 
15 program codes are stored in a memory provided in a 
function expansion board in the computer or a function 
expansion unit connected to the computer, and 
subsequently the CPU or the like provided in the 
function expansion board or the function expansion unit 
20 performs part or all of actual processes based on 
instructions of the program codes, and the above, 
described embodiments are realized by those processes, 
is also included in the category of the present 
invention. 
25 [0056] 

If the present invention is applied to the above 
described storage medium, the program codes 
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corresponding to the flowcharts (shown in Figure 3 
and/or Figure 11, and/or Figure 12, and/or Figure 19) 
described above are stored in the storage medium. 
[0057] 

5 [Effect of the Invention] 

As described above, according to the present 
invention, the compression ratio of the image can be 
improved by reducing the number of the colors in the 
color-reduced image. 
10 [Brief Description of the Drawings] 
[Figure 1] 

Figure 1 is a block diagram showing a 
configuration of an image processing apparatus for 
generating a compressed data 1A which will be described 
15 below, and processes (processing results) in its 
respective units in a first embodiment of the present 
invention. 
[Figure 2] 

Figure 2 is a block diagram showing a 
20 configuration of the image processing apparatus for 
decompressing the compressed data 1A and also processes 
in its respective units in the first embodiment of the 
present invention. 
[Figure 3] 

25 Figure 3 is a flowchart of a process until a 

character area is extracted from an original image 101. 
[Figure 4] 
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Figure 4 is a diagram showing a color manuscript 
used in the first embodiment of the present invention. 
[Figure 5] 

Figure 5 is a histogram of a luminance value of 
the color manuscript shown in Figure 4. 
[Figure 6] 

Figure 6 is a diagram showing an image in which 
the color manuscript shown in Figure 4 has been 
binarized. 
[Figure 7] 

Figure 7 is a diagram showing an image in the 
case where a contour tracing is performed with respect 
to black pixels of the image shown in Figure 6 to label 
all, and for example, only collections of the black 
pixels having widths less than or equal to a 
predetermined threshold or heights less than or equal 
to a predetermined threshold have been permitted as 
characters . 
[Figure 8] 

Figure 8 is a diagram showing an image in the 
case where the collections of the black pixels have 
been grouped in accordance with positional proximity or 
matching widths or heights in the image shown in Figure 
7. 

[Figure 9] 

Figure 9 is a histogram for describing luminance 
values of ground colors and character colors in a 
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second embodiment of the present invention. 
[Figure 10] 

Figure 10 (a) is a diagram showing an original 
image, Figure 10 {b) is a diagram showing a binary image 
of a character area, and Figure 10 {c) is a diagram 
showing how the original image shown in (a) has been 
segmented into parts. 
[Figure 11] 

Figure 11 is a flowchart of a process in a 
character portion filling unit 105 by using the 
original image and a full-page binarized image (binary 
image) . 
[Figure 12] 

Figure 12 is a flowchart of a process in a 
character color extraction unit 108. 
[Figure 13] 

Figure 13 is a diagram showing an original 
maximum value across two histograms. 
[Figure 14] 

Figure 14(a) is a diagram showing a result of 
JPEG-decompressing a compression code X, Figure 14(b) 
is a diagram showing a result of decompressing 
compression codes Y (binary image) , and Figure 14 (c) is 
a diagram showing an image in which data on the image 
shown in (b) is put on the image shown in (a) . 
[Figure 15] 

Figure 15 is a diagram for describing a square 
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for detecting a maximum value in an RGB 3D histogram. 
[Figure 16] 

Figure 16 is a block diagram showing a 
configuration of the image processing apparatus in a 
5 second embodiment of the present invention. 
[Figure 17] 

Figure 17{a) is a diagram showing a linear 
differential filter which detects vertical lines and 
horizontal lines, and Figure 17 (b) is a diagram showing 
10 a quadratic differential filter which accommodates all 
directions. 
[Figure 18] 

Figures 18(a) to (cO) are diagrams showing 
luminance distributions of three patterns of an image 
15 and a ground image of the character area. 
[Figure 19] 

Figure 19 is a flowchart of a process in a color 
reduction unit 1082 within the character color 
extraction unit 108. 

20 
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Figure 3 

S301 PERFORM LUMINANCE CONVERSION WITH RESPECT TO 

INPUTTED COLOR IMAGE WHILE THINNING OUT IMAGE r AND 
GENERATE LUMINANCE IMAGE J 
20 S302 FORM HISTOGRAM OF LUMINANCE AND CALCULATE 

BINARIZATION THRESHOLD T 

5303 BINARIZE IMAGE J WITH T AND GENERATE FULL-PAGE 
BINARIZED IMAGE 103 

5304 PERFORM CONTOUR TRACING WITH RESPECT TO BLACK 
25 PIXELS AND LABEL ALL BLACK AREAS 

53 05 DETERMINE CHARACTER-LIKE AREAS IN BLACK AREAS 

53 06 INTEGRATE AREAS WHICH SHOULD BE INTEGRATED TO 
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EACH OTHER IN ACCORDANCE WITH SHAPES OR POSITIONS 
Figure 4 

#1 xxxxxxxxxxxx at the personal computer on your 

5 desk 

#2 Personal computer banking will be familiar to 

you 

Figure 6 

10 #1 xxxxxxxxxxxx at the personal computer on your 

desk 

#2 Personal computer banking will be familiar to 

you 

15 Figure 7 

#1 xxxxxxxxxxxx at the personal computer on your 
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#2 Personal computer banking will be familiar to 

you 

20 

Figure 9 
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Figure 10 
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#4 BLACK 

#5 BINARY IMAGE 

#6 COLOR OF INSIDE ABC BECOMES AVERAGE VALUE 

COLOR OF PIXELS OTHER THAN ITS BLOCK CHARACTER FOR EACH 
5 BLOCK 

Figure 11 

51101 SEGMENT IMAGE INTO 32x32 AREAS (PARTS) 

51102 IS THERE ANY UNPROCESSED PART? 
10 S1103 IS THERE ANY CHARACTER AREA? . 

51104 WITH REFERENCE TO CORRESPONDING BINARIZED 
IMAGE, CALCULATE AVERAGE COLOR ave_color OF WHITE PIXEL 
PORTION 

51105 WITH REFERENCE TO CORRESPONDING BINARY IMAGE, 
15 ASSIGN ave_color TO BLACK PIXEL PORTION 

Figure 12 

S1202 ARE THERE ANY UNPROCESSED CHARACTER AREA 
COORDINATES? 

20 S1203 PERFORM THINNING PROCESS WITH RESPECT TO 
BINARY IMAGE AND GENERATE NEW BINARY IMAGE newbi 

51204 WITH REFERENCE TO Newbi, FORM RGB 3D HISTOGRAM 
OF ORIGINAL IMAGE CORRESPONDING TO BLACK PIXELS 

ON THIS OCCASION, ALSO CALCULATE NUMBER OF PIXELS 
25 Pixelnum 

51205 ASSIGN MAXIMUM VALUE FROM HISTOGRAM TO 
colR[num], colG[num], B[num] 
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S1208 SET HISTOGRAM VALUES POSITIONED WITHIN SQUARE 
WHICH HAS BEEN ENLARGED BY haba CENTERING AROUND 
MAXIMUM VALUE FROM RGB HISTOGRAM, TO 0 

ON THIS OCCASION, SUBTRACT NUMBER OF PIXELS FROM 
5 pixelnum 



Figure 14 

#1 REPRESENTATIVE COLOR DATA R=20, G=30, B=255 

#2 BLUE OF (20, 30, 255) 

Figure 15 

#1 WHITE 

#2 7 Steps CENTERING AROUND EXTRACTED POINT 

#3 BLACK 

Figure 16 

17 01 ORIGINAL IMAGE 

1702 DIFFERENTIAL PROCESSING UNIT 

1703 DIFFERENTIALLY BINARIZED IMAGE 

1704 CHARACTER AREA DETECTION UNIT 

1705 BINARIZATION UNIT 

1706 PARTIAL BINARY IMAGE 

1707 CHARACTER PORTION FILLING UNIT 

1708 REDUCTION UNIT 

1709 JPEG COMPRESSION UNIT 

1710 CHARACTER COLOR EXTRACTION UNIT 

1711 COLOR-REDUCED IMAGES 
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1712 MMR COMPRESSION UNIT 

1713 ZIP COMPRESSION UNIT 

1714 CHARACTER AREA COORDINATES 

1715 COMPRESSION CODE X 
5 1716 PALETTES 

1717 COMPRESSION CODES Y 

1718 COMPRESSION CODES Z 

#1 NUMBER OF BITS IN IMAGE m 

#2 PALETTES 

10 #3 m-BIT IMAGE 

#4 COLOR REDUCTION UNIT 

#5 WHEN =1 bit, EQUAL PORTION IS BINARY IMAGE 

Figure 18 

15 #1 CHARACTER 

#2 GROUND 

#3 LUMINANCE 

Figure 19 

20 S1901 NUMBER OF PALETTES? 

S1902 SORT VALUE IN ORIGINAL IMAGE OF PIXEL WHICH IS 



BLACK IN BINARY IMAGE WITHIN CHARACTER AREA, INTO VALUE 
OF PALETTE, AND GENERATE COLOR- REDUCED IMAGE 
SI 9 03 EXTRACT REPRESENTATIVE COLOR FOR EACH ONE 
25 CHARACTER BY USING CHARACTER COORDINATES OF EACH ONE 
CHARACTER 

SI 904 REPRESENT EACH CHARACTER IN REPRESENTATIVE 
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COLOR (GENERATE COLOR-REDUCED IMAGE AGAIN) 

51905 DELETE PALETTE COLOR WHICH HAS NOT BEEN USED 

51 90 6 HAS PROCESS BEEN PERFORMED WITH RESPECT TO ALL 
CHARACTER AREAS? 

5 SI 90 7 GENERATE ONE- PORT ION BINARY IMAGE 
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